{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "vscode": {
     "languageId": "markdown"
    }
   },
   "source": [
    "# Feedback Loop in RAG\n",
    "\n",
    "In this notebook, I implement a RAG system with a feedback loop mechanism that continuously improves over time. By collecting and incorporating user feedback, our system learns to provide more relevant and higher-quality responses with each interaction.\n",
    "\n",
    "Traditional RAG systems are static - they retrieve information based solely on embedding similarity. With a feedback loop, we create a dynamic system that:\n",
    "\n",
    "- Remembers what worked (and what didn't)\n",
    "- Adjusts document relevance scores over time\n",
    "- Incorporates successful Q&A pairs into its knowledge base\n",
    "- Gets smarter with each user interaction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting Up the Environment\n",
    "We begin by importing necessary libraries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import fitz\n",
    "import os\n",
    "import numpy as np\n",
    "import json\n",
    "from openai import OpenAI\n",
    "from datetime import datetime"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Extracting Text from a PDF File\n",
    "To implement RAG, we first need a source of textual data. In this case, we extract text from a PDF file using the PyMuPDF library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def extract_text_from_pdf(pdf_path):\n",
    "    \"\"\"\n",
    "    Extracts text from a PDF file and prints the first `num_chars` characters.\n",
    "\n",
    "    Args:\n",
    "    pdf_path (str): Path to the PDF file.\n",
    "\n",
    "    Returns:\n",
    "    str: Extracted text from the PDF.\n",
    "    \"\"\"\n",
    "    # Open the PDF file\n",
    "    mypdf = fitz.open(pdf_path)\n",
    "    all_text = \"\"  # Initialize an empty string to store the extracted text\n",
    "\n",
    "    # Iterate through each page in the PDF\n",
    "    for page_num in range(mypdf.page_count):\n",
    "        page = mypdf[page_num]  # Get the page\n",
    "        text = page.get_text(\"text\")  # Extract text from the page\n",
    "        all_text += text  # Append the extracted text to the all_text string\n",
    "\n",
    "    return all_text  # Return the extracted text"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Chunking the Extracted Text\n",
    "Once we have the extracted text, we divide it into smaller, overlapping chunks to improve retrieval accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def chunk_text(text, n, overlap):\n",
    "    \"\"\"\n",
    "    Chunks the given text into segments of n characters with overlap.\n",
    "\n",
    "    Args:\n",
    "    text (str): The text to be chunked.\n",
    "    n (int): The number of characters in each chunk.\n",
    "    overlap (int): The number of overlapping characters between chunks.\n",
    "\n",
    "    Returns:\n",
    "    List[str]: A list of text chunks.\n",
    "    \"\"\"\n",
    "    chunks = []  # Initialize an empty list to store the chunks\n",
    "    \n",
    "    # Loop through the text with a step size of (n - overlap)\n",
    "    for i in range(0, len(text), n - overlap):\n",
    "        # Append a chunk of text from index i to i + n to the chunks list\n",
    "        chunks.append(text[i:i + n])\n",
    "\n",
    "    return chunks  # Return the list of text chunks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setting Up the OpenAI API Client\n",
    "We initialize the OpenAI client to generate embeddings and responses."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initialize the OpenAI client with the base URL and API key\n",
    "client = OpenAI(\n",
    "    base_url=\"https://api.studio.nebius.com/v1/\",\n",
    "    api_key=os.getenv(\"OPENAI_API_KEY\")  # Retrieve the API key from environment variables\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Simple Vector Store Implementation\n",
    "We'll create a basic vector store to manage document chunks and their embeddings."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "class SimpleVectorStore:\n",
    "    \"\"\"\n",
    "    A simple vector store implementation using NumPy.\n",
    "    \n",
    "    This class provides an in-memory storage and retrieval system for \n",
    "    embedding vectors and their corresponding text chunks and metadata.\n",
    "    It supports basic similarity search functionality using cosine similarity.\n",
    "    \"\"\"\n",
    "    def __init__(self):\n",
    "        \"\"\"\n",
    "        Initialize the vector store with empty lists for vectors, texts, and metadata.\n",
    "        \n",
    "        The vector store maintains three parallel lists:\n",
    "        - vectors: NumPy arrays of embedding vectors\n",
    "        - texts: Original text chunks corresponding to each vector\n",
    "        - metadata: Optional metadata dictionaries for each item\n",
    "        \"\"\"\n",
    "        self.vectors = []  # List to store embedding vectors\n",
    "        self.texts = []    # List to store original text chunks\n",
    "        self.metadata = [] # List to store metadata for each text chunk\n",
    "    \n",
    "    def add_item(self, text, embedding, metadata=None):\n",
    "        \"\"\"\n",
    "        Add an item to the vector store.\n",
    "\n",
    "        Args:\n",
    "            text (str): The original text chunk to store.\n",
    "            embedding (List[float]): The embedding vector representing the text.\n",
    "            metadata (dict, optional): Additional metadata for the text chunk,\n",
    "                                      such as source, timestamp, or relevance scores.\n",
    "        \"\"\"\n",
    "        self.vectors.append(np.array(embedding))  # Convert and store the embedding\n",
    "        self.texts.append(text)                   # Store the original text\n",
    "        self.metadata.append(metadata or {})      # Store metadata (empty dict if None)\n",
    "    \n",
    "    def similarity_search(self, query_embedding, k=5, filter_func=None):\n",
    "        \"\"\"\n",
    "        Find the most similar items to a query embedding using cosine similarity.\n",
    "\n",
    "        Args:\n",
    "            query_embedding (List[float]): Query embedding vector to compare against stored vectors.\n",
    "            k (int): Number of most similar results to return.\n",
    "            filter_func (callable, optional): Function to filter results based on metadata.\n",
    "                                             Takes metadata dict as input and returns boolean.\n",
    "\n",
    "        Returns:\n",
    "            List[Dict]: Top k most similar items, each containing:\n",
    "                - text: The original text\n",
    "                - metadata: Associated metadata\n",
    "                - similarity: Raw cosine similarity score\n",
    "                - relevance_score: Either metadata-based relevance or calculated similarity\n",
    "                \n",
    "        Note: Returns empty list if no vectors are stored or none pass the filter.\n",
    "        \"\"\"\n",
    "        if not self.vectors:\n",
    "            return []  # Return empty list if vector store is empty\n",
    "        \n",
    "        # Convert query embedding to numpy array for vector operations\n",
    "        query_vector = np.array(query_embedding)\n",
    "        \n",
    "        # Calculate cosine similarity between query and each stored vector\n",
    "        similarities = []\n",
    "        for i, vector in enumerate(self.vectors):\n",
    "            # Skip items that don't pass the filter criteria\n",
    "            if filter_func and not filter_func(self.metadata[i]):\n",
    "                continue\n",
    "                \n",
    "            # Calculate cosine similarity: dot product / (norm1 * norm2)\n",
    "            similarity = np.dot(query_vector, vector) / (np.linalg.norm(query_vector) * np.linalg.norm(vector))\n",
    "            similarities.append((i, similarity))  # Store index and similarity score\n",
    "        \n",
    "        # Sort results by similarity score in descending order\n",
    "        similarities.sort(key=lambda x: x[1], reverse=True)\n",
    "        \n",
    "        # Construct result dictionaries for the top k matches\n",
    "        results = []\n",
    "        for i in range(min(k, len(similarities))):\n",
    "            idx, score = similarities[i]\n",
    "            results.append({\n",
    "                \"text\": self.texts[idx],\n",
    "                \"metadata\": self.metadata[idx],\n",
    "                \"similarity\": score,\n",
    "                # Use pre-existing relevance score from metadata if available, otherwise use similarity\n",
    "                \"relevance_score\": self.metadata[idx].get(\"relevance_score\", score)\n",
    "            })\n",
    "        \n",
    "        return results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_embeddings(text, model=\"BAAI/bge-en-icl\"):\n",
    "    \"\"\"\n",
    "    Creates embeddings for the given text.\n",
    "\n",
    "    Args:\n",
    "    text (str or List[str]): The input text(s) for which embeddings are to be created.\n",
    "    model (str): The model to be used for creating embeddings.\n",
    "\n",
    "    Returns:\n",
    "    List[float] or List[List[float]]: The embedding vector(s).\n",
    "    \"\"\"\n",
    "    # Convert single string to list for uniform processing\n",
    "    input_text = text if isinstance(text, list) else [text]\n",
    "    \n",
    "    # Call the OpenAI API to generate embeddings for all input texts\n",
    "    response = client.embeddings.create(\n",
    "        model=model,\n",
    "        input=input_text\n",
    "    )\n",
    "    \n",
    "    # For single string input, return just the first embedding vector\n",
    "    if isinstance(text, str):\n",
    "        return response.data[0].embedding\n",
    "    \n",
    "    # For list input, return a list of all embedding vectors\n",
    "    return [item.embedding for item in response.data]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Feedback System Functions\n",
    "Now we'll implement the core feedback system components."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_user_feedback(query, response, relevance, quality, comments=\"\"):\n",
    "    \"\"\"\n",
    "    Format user feedback in a dictionary.\n",
    "    \n",
    "    Args:\n",
    "        query (str): User's query\n",
    "        response (str): System's response\n",
    "        relevance (int): Relevance score (1-5)\n",
    "        quality (int): Quality score (1-5)\n",
    "        comments (str): Optional feedback comments\n",
    "        \n",
    "    Returns:\n",
    "        Dict: Formatted feedback\n",
    "    \"\"\"\n",
    "    return {\n",
    "        \"query\": query,\n",
    "        \"response\": response,\n",
    "        \"relevance\": int(relevance),\n",
    "        \"quality\": int(quality),\n",
    "        \"comments\": comments,\n",
    "        \"timestamp\": datetime.now().isoformat()\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def store_feedback(feedback, feedback_file=\"feedback_data.json\"):\n",
    "    \"\"\"\n",
    "    Store feedback in a JSON file.\n",
    "    \n",
    "    Args:\n",
    "        feedback (Dict): Feedback data\n",
    "        feedback_file (str): Path to feedback file\n",
    "    \"\"\"\n",
    "    with open(feedback_file, \"a\") as f:\n",
    "        json.dump(feedback, f)\n",
    "        f.write(\"\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_feedback_data(feedback_file=\"feedback_data.json\"):\n",
    "    \"\"\"\n",
    "    Load feedback data from file.\n",
    "    \n",
    "    Args:\n",
    "        feedback_file (str): Path to feedback file\n",
    "        \n",
    "    Returns:\n",
    "        List[Dict]: List of feedback entries\n",
    "    \"\"\"\n",
    "    feedback_data = []\n",
    "    try:\n",
    "        with open(feedback_file, \"r\") as f:\n",
    "            for line in f:\n",
    "                if line.strip():\n",
    "                    feedback_data.append(json.loads(line.strip()))\n",
    "    except FileNotFoundError:\n",
    "        print(\"No feedback data file found. Starting with empty feedback.\")\n",
    "    \n",
    "    return feedback_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Document Processing with Feedback Awareness"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "def process_document(pdf_path, chunk_size=1000, chunk_overlap=200):\n",
    "    \"\"\"\n",
    "    Process a document for RAG (Retrieval Augmented Generation) with feedback loop.\n",
    "    This function handles the complete document processing pipeline:\n",
    "    1. Text extraction from PDF\n",
    "    2. Text chunking with overlap\n",
    "    3. Embedding creation for chunks\n",
    "    4. Storage in vector database with metadata\n",
    "\n",
    "    Args:\n",
    "    pdf_path (str): Path to the PDF file to process.\n",
    "    chunk_size (int): Size of each text chunk in characters.\n",
    "    chunk_overlap (int): Number of overlapping characters between consecutive chunks.\n",
    "\n",
    "    Returns:\n",
    "    Tuple[List[str], SimpleVectorStore]: A tuple containing:\n",
    "        - List of document chunks\n",
    "        - Populated vector store with embeddings and metadata\n",
    "    \"\"\"\n",
    "    # Step 1: Extract raw text content from the PDF document\n",
    "    print(\"Extracting text from PDF...\")\n",
    "    extracted_text = extract_text_from_pdf(pdf_path)\n",
    "    \n",
    "    # Step 2: Split text into manageable, overlapping chunks for better context preservation\n",
    "    print(\"Chunking text...\")\n",
    "    chunks = chunk_text(extracted_text, chunk_size, chunk_overlap)\n",
    "    print(f\"Created {len(chunks)} text chunks\")\n",
    "    \n",
    "    # Step 3: Generate vector embeddings for each text chunk\n",
    "    print(\"Creating embeddings for chunks...\")\n",
    "    chunk_embeddings = create_embeddings(chunks)\n",
    "    \n",
    "    # Step 4: Initialize the vector database to store chunks and their embeddings\n",
    "    store = SimpleVectorStore()\n",
    "    \n",
    "    # Step 5: Add each chunk with its embedding to the vector store\n",
    "    # Include metadata for feedback-based improvements\n",
    "    for i, (chunk, embedding) in enumerate(zip(chunks, chunk_embeddings)):\n",
    "        store.add_item(\n",
    "            text=chunk,\n",
    "            embedding=embedding,\n",
    "            metadata={\n",
    "                \"index\": i,                # Position in original document\n",
    "                \"source\": pdf_path,        # Source document path\n",
    "                \"relevance_score\": 1.0,    # Initial relevance score (will be updated with feedback)\n",
    "                \"feedback_count\": 0        # Counter for feedback received on this chunk\n",
    "            }\n",
    "        )\n",
    "    \n",
    "    print(f\"Added {len(chunks)} chunks to the vector store\")\n",
    "    return chunks, store"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Relevance Adjustment Based on Feedback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def assess_feedback_relevance(query, doc_text, feedback):\n",
    "    \"\"\"\n",
    "    Use LLM to assess if a past feedback entry is relevant to the current query and document.\n",
    "    \n",
    "    This function helps determine which past feedback should influence the current retrieval\n",
    "    by sending the current query, past query+feedback, and document content to an LLM\n",
    "    for relevance assessment.\n",
    "    \n",
    "    Args:\n",
    "        query (str): Current user query that needs information retrieval\n",
    "        doc_text (str): Text content of the document being evaluated\n",
    "        feedback (Dict): Previous feedback data containing 'query' and 'response' keys\n",
    "        \n",
    "    Returns:\n",
    "        bool: True if the feedback is deemed relevant to current query/document, False otherwise\n",
    "    \"\"\"\n",
    "    # Define system prompt instructing the LLM to make binary relevance judgments only\n",
    "    system_prompt = \"\"\"You are an AI system that determines if a past feedback is relevant to a current query and document.\n",
    "    Answer with ONLY 'yes' or 'no'. Your job is strictly to determine relevance, not to provide explanations.\"\"\"\n",
    "\n",
    "    # Construct user prompt with current query, past feedback data, and truncated document content\n",
    "    user_prompt = f\"\"\"\n",
    "    Current query: {query}\n",
    "    Past query that received feedback: {feedback['query']}\n",
    "    Document content: {doc_text[:500]}... [truncated]\n",
    "    Past response that received feedback: {feedback['response'][:500]}... [truncated]\n",
    "\n",
    "    Is this past feedback relevant to the current query and document? (yes/no)\n",
    "    \"\"\"\n",
    "\n",
    "    # Call the LLM API with zero temperature for deterministic output\n",
    "    response = client.chat.completions.create(\n",
    "        model=\"meta-llama/Llama-3.2-3B-Instruct\",\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": system_prompt},\n",
    "            {\"role\": \"user\", \"content\": user_prompt}\n",
    "        ],\n",
    "        temperature=0  # Use temperature=0 for consistent, deterministic responses\n",
    "    )\n",
    "    \n",
    "    # Extract and normalize the response to determine relevance\n",
    "    answer = response.choices[0].message.content.strip().lower()\n",
    "    return 'yes' in answer  # Return True if the answer contains 'yes'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def adjust_relevance_scores(query, results, feedback_data):\n",
    "    \"\"\"\n",
    "    Adjust document relevance scores based on historical feedback to improve retrieval quality.\n",
    "    \n",
    "    This function analyzes past user feedback to dynamically adjust the relevance scores of \n",
    "    retrieved documents. It identifies feedback that is relevant to the current query context,\n",
    "    calculates score modifiers based on relevance ratings, and re-ranks the results accordingly.\n",
    "    \n",
    "    Args:\n",
    "        query (str): Current user query\n",
    "        results (List[Dict]): Retrieved documents with their original similarity scores\n",
    "        feedback_data (List[Dict]): Historical feedback containing user ratings\n",
    "        \n",
    "    Returns:\n",
    "        List[Dict]: Results with adjusted relevance scores, sorted by the new scores\n",
    "    \"\"\"\n",
    "    # If no feedback data available, return original results unchanged\n",
    "    if not feedback_data:\n",
    "        return results\n",
    "    \n",
    "    print(\"Adjusting relevance scores based on feedback history...\")\n",
    "    \n",
    "    # Process each retrieved document\n",
    "    for i, result in enumerate(results):\n",
    "        document_text = result[\"text\"]\n",
    "        relevant_feedback = []\n",
    "        \n",
    "        # Find relevant feedback for this specific document and query combination\n",
    "        # by querying the LLM to assess relevance of each historical feedback item\n",
    "        for feedback in feedback_data:\n",
    "            is_relevant = assess_feedback_relevance(query, document_text, feedback)\n",
    "            if is_relevant:\n",
    "                relevant_feedback.append(feedback)\n",
    "        \n",
    "        # Apply score adjustments if relevant feedback exists\n",
    "        if relevant_feedback:\n",
    "            # Calculate average relevance rating from all applicable feedback entries\n",
    "            # Feedback relevance is on a 1-5 scale (1=not relevant, 5=highly relevant)\n",
    "            avg_relevance = sum(f['relevance'] for f in relevant_feedback) / len(relevant_feedback)\n",
    "            \n",
    "            # Convert the average relevance to a score modifier in range 0.5-1.5\n",
    "            # - Scores below 3/5 will reduce the original similarity (modifier < 1.0)\n",
    "            # - Scores above 3/5 will increase the original similarity (modifier > 1.0)\n",
    "            modifier = 0.5 + (avg_relevance / 5.0)\n",
    "            \n",
    "            # Apply the modifier to the original similarity score\n",
    "            original_score = result[\"similarity\"]\n",
    "            adjusted_score = original_score * modifier\n",
    "            \n",
    "            # Update the result dictionary with new scores and feedback metadata\n",
    "            result[\"original_similarity\"] = original_score  # Preserve the original score\n",
    "            result[\"similarity\"] = adjusted_score           # Update the primary score\n",
    "            result[\"relevance_score\"] = adjusted_score      # Update the relevance score\n",
    "            result[\"feedback_applied\"] = True               # Flag that feedback was applied\n",
    "            result[\"feedback_count\"] = len(relevant_feedback)  # Number of feedback entries used\n",
    "            \n",
    "            # Log the adjustment details\n",
    "            print(f\"  Document {i+1}: Adjusted score from {original_score:.4f} to {adjusted_score:.4f} based on {len(relevant_feedback)} feedback(s)\")\n",
    "    \n",
    "    # Re-sort results by adjusted scores to ensure higher quality matches appear first\n",
    "    results.sort(key=lambda x: x[\"similarity\"], reverse=True)\n",
    "    \n",
    "    return results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Fine-tuning Our Index with Feedback"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "def fine_tune_index(current_store, chunks, feedback_data):\n",
    "    \"\"\"\n",
    "    Enhance vector store with high-quality feedback to improve retrieval quality over time.\n",
    "    \n",
    "    This function implements a continuous learning process by:\n",
    "    1. Identifying high-quality feedback (highly rated Q&A pairs)\n",
    "    2. Creating new retrieval items from successful interactions\n",
    "    3. Adding these to the vector store with boosted relevance weights\n",
    "    \n",
    "    Args:\n",
    "        current_store (SimpleVectorStore): Current vector store containing original document chunks\n",
    "        chunks (List[str]): Original document text chunks \n",
    "        feedback_data (List[Dict]): Historical user feedback with relevance and quality ratings\n",
    "        \n",
    "    Returns:\n",
    "        SimpleVectorStore: Enhanced vector store containing both original chunks and feedback-derived content\n",
    "    \"\"\"\n",
    "    print(\"Fine-tuning index with high-quality feedback...\")\n",
    "    \n",
    "    # Filter for only high-quality responses (both relevance and quality rated 4 or 5)\n",
    "    # This ensures we only learn from the most successful interactions\n",
    "    good_feedback = [f for f in feedback_data if f['relevance'] >= 4 and f['quality'] >= 4]\n",
    "    \n",
    "    if not good_feedback:\n",
    "        print(\"No high-quality feedback found for fine-tuning.\")\n",
    "        return current_store  # Return original store unchanged if no good feedback exists\n",
    "    \n",
    "    # Initialize new store that will contain both original and enhanced content\n",
    "    new_store = SimpleVectorStore()\n",
    "    \n",
    "    # First transfer all original document chunks with their existing metadata\n",
    "    for i in range(len(current_store.texts)):\n",
    "        new_store.add_item(\n",
    "            text=current_store.texts[i],\n",
    "            embedding=current_store.vectors[i],\n",
    "            metadata=current_store.metadata[i].copy()  # Use copy to prevent reference issues\n",
    "        )\n",
    "    \n",
    "    # Create and add enhanced content from good feedback\n",
    "    for feedback in good_feedback:\n",
    "        # Format a new document that combines the question and its high-quality answer\n",
    "        # This creates retrievable content that directly addresses user queries\n",
    "        enhanced_text = f\"Question: {feedback['query']}\\nAnswer: {feedback['response']}\"\n",
    "        \n",
    "        # Generate embedding vector for this new synthetic document\n",
    "        embedding = create_embeddings(enhanced_text)\n",
    "        \n",
    "        # Add to vector store with special metadata that identifies its origin and importance\n",
    "        new_store.add_item(\n",
    "            text=enhanced_text,\n",
    "            embedding=embedding,\n",
    "            metadata={\n",
    "                \"type\": \"feedback_enhanced\",  # Mark as derived from feedback\n",
    "                \"query\": feedback[\"query\"],   # Store original query for reference\n",
    "                \"relevance_score\": 1.2,       # Boost initial relevance to prioritize these items\n",
    "                \"feedback_count\": 1,          # Track feedback incorporation\n",
    "                \"original_feedback\": feedback # Preserve complete feedback record\n",
    "            }\n",
    "        )\n",
    "        \n",
    "        print(f\"Added enhanced content from feedback: {feedback['query'][:50]}...\")\n",
    "    \n",
    "    # Log summary statistics about the enhancement\n",
    "    print(f\"Fine-tuned index now has {len(new_store.texts)} items (original: {len(chunks)})\")\n",
    "    return new_store"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complete RAG Pipeline with Feedback Loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_response(query, context, model=\"meta-llama/Llama-3.2-3B-Instruct\"):\n",
    "    \"\"\"\n",
    "    Generate a response based on the query and context.\n",
    "    \n",
    "    Args:\n",
    "        query (str): User query\n",
    "        context (str): Context text from retrieved documents\n",
    "        model (str): LLM model to use\n",
    "        \n",
    "    Returns:\n",
    "        str: Generated response\n",
    "    \"\"\"\n",
    "    # Define the system prompt to guide the AI's behavior\n",
    "    system_prompt = \"\"\"You are a helpful AI assistant. Answer the user's question based only on the provided context. If you cannot find the answer in the context, state that you don't have enough information.\"\"\"\n",
    "    \n",
    "    # Create the user prompt by combining the context and the query\n",
    "    user_prompt = f\"\"\"\n",
    "        Context:\n",
    "        {context}\n",
    "\n",
    "        Question: {query}\n",
    "\n",
    "        Please provide a comprehensive answer based only on the context above.\n",
    "    \"\"\"\n",
    "    \n",
    "    # Call the OpenAI API to generate a response based on the system and user prompts\n",
    "    response = client.chat.completions.create(\n",
    "        model=model,\n",
    "        messages=[\n",
    "            {\"role\": \"system\", \"content\": system_prompt},\n",
    "            {\"role\": \"user\", \"content\": user_prompt}\n",
    "        ],\n",
    "        temperature=0  # Use temperature=0 for consistent, deterministic responses\n",
    "    )\n",
    "    \n",
    "    # Return the generated response content\n",
    "    return response.choices[0].message.content"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "def rag_with_feedback_loop(query, vector_store, feedback_data, k=5, model=\"meta-llama/Llama-3.2-3B-Instruct\"):\n",
    "    \"\"\"\n",
    "    Complete RAG pipeline incorporating feedback loop.\n",
    "    \n",
    "    Args:\n",
    "        query (str): User query\n",
    "        vector_store (SimpleVectorStore): Vector store with document chunks\n",
    "        feedback_data (List[Dict]): History of feedback\n",
    "        k (int): Number of documents to retrieve\n",
    "        model (str): LLM model for response generation\n",
    "        \n",
    "    Returns:\n",
    "        Dict: Results including query, retrieved documents, and response\n",
    "    \"\"\"\n",
    "    print(f\"\\n=== Processing query with feedback-enhanced RAG ===\")\n",
    "    print(f\"Query: {query}\")\n",
    "    \n",
    "    # Step 1: Create query embedding\n",
    "    query_embedding = create_embeddings(query)\n",
    "    \n",
    "    # Step 2: Perform initial retrieval based on query embedding\n",
    "    results = vector_store.similarity_search(query_embedding, k=k)\n",
    "    \n",
    "    # Step 3: Adjust relevance scores of retrieved documents based on feedback\n",
    "    adjusted_results = adjust_relevance_scores(query, results, feedback_data)\n",
    "    \n",
    "    # Step 4: Extract texts from adjusted results for context building\n",
    "    retrieved_texts = [result[\"text\"] for result in adjusted_results]\n",
    "    \n",
    "    # Step 5: Build context for response generation by concatenating retrieved texts\n",
    "    context = \"\\n\\n---\\n\\n\".join(retrieved_texts)\n",
    "    \n",
    "    # Step 6: Generate response using the context and query\n",
    "    print(\"Generating response...\")\n",
    "    response = generate_response(query, context, model)\n",
    "    \n",
    "    # Step 7: Compile the final result\n",
    "    result = {\n",
    "        \"query\": query,\n",
    "        \"retrieved_documents\": adjusted_results,\n",
    "        \"response\": response\n",
    "    }\n",
    "    \n",
    "    print(\"\\n=== Response ===\")\n",
    "    print(response)\n",
    "    \n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Complete Workflow: From Initial Setup to Feedback Collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "def full_rag_workflow(pdf_path, query, feedback_data=None, feedback_file=\"feedback_data.json\", fine_tune=False):\n",
    "    \"\"\"\n",
    "    Execute a complete RAG workflow with feedback integration for continuous improvement.\n",
    "    \n",
    "    This function orchestrates the entire Retrieval-Augmented Generation process:\n",
    "    1. Load historical feedback data\n",
    "    2. Process and chunk the document\n",
    "    3. Optionally fine-tune the vector index with prior feedback\n",
    "    4. Perform retrieval and generation with feedback-adjusted relevance scores\n",
    "    5. Collect new user feedback for future improvement\n",
    "    6. Store feedback to enable system learning over time\n",
    "    \n",
    "    Args:\n",
    "        pdf_path (str): Path to the PDF document to be processed\n",
    "        query (str): User's natural language query\n",
    "        feedback_data (List[Dict], optional): Pre-loaded feedback data, loads from file if None\n",
    "        feedback_file (str): Path to the JSON file storing feedback history\n",
    "        fine_tune (bool): Whether to enhance the index with successful past Q&A pairs\n",
    "        \n",
    "    Returns:\n",
    "        Dict: Results containing the response and retrieval metadata\n",
    "    \"\"\"\n",
    "    # Step 1: Load historical feedback for relevance adjustment if not explicitly provided\n",
    "    if feedback_data is None:\n",
    "        feedback_data = load_feedback_data(feedback_file)\n",
    "        print(f\"Loaded {len(feedback_data)} feedback entries from {feedback_file}\")\n",
    "    \n",
    "    # Step 2: Process document through extraction, chunking and embedding pipeline\n",
    "    chunks, vector_store = process_document(pdf_path)\n",
    "    \n",
    "    # Step 3: Fine-tune the vector index by incorporating high-quality past interactions\n",
    "    # This creates enhanced retrievable content from successful Q&A pairs\n",
    "    if fine_tune and feedback_data:\n",
    "        vector_store = fine_tune_index(vector_store, chunks, feedback_data)\n",
    "    \n",
    "    # Step 4: Execute core RAG with feedback-aware retrieval\n",
    "    # Note: This depends on the rag_with_feedback_loop function which should be defined elsewhere\n",
    "    result = rag_with_feedback_loop(query, vector_store, feedback_data)\n",
    "    \n",
    "    # Step 5: Collect user feedback to improve future performance\n",
    "    print(\"\\n=== Would you like to provide feedback on this response? ===\")\n",
    "    print(\"Rate relevance (1-5, with 5 being most relevant):\")\n",
    "    relevance = input()\n",
    "    \n",
    "    print(\"Rate quality (1-5, with 5 being highest quality):\")\n",
    "    quality = input()\n",
    "    \n",
    "    print(\"Any comments? (optional, press Enter to skip)\")\n",
    "    comments = input()\n",
    "    \n",
    "    # Step 6: Format feedback into structured data\n",
    "    feedback = get_user_feedback(\n",
    "        query=query,\n",
    "        response=result[\"response\"],\n",
    "        relevance=int(relevance),\n",
    "        quality=int(quality),\n",
    "        comments=comments\n",
    "    )\n",
    "    \n",
    "    # Step 7: Persist feedback to enable continuous system learning\n",
    "    store_feedback(feedback, feedback_file)\n",
    "    print(\"Feedback recorded. Thank you!\")\n",
    "    \n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluating Our Feedback Loop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def evaluate_feedback_loop(pdf_path, test_queries, reference_answers=None):\n",
    "    \"\"\"\n",
    "    Evaluate the impact of feedback loop on RAG quality by comparing performance before and after feedback integration.\n",
    "    \n",
    "    This function runs a controlled experiment to measure how incorporating feedback affects retrieval and generation:\n",
    "    1. First round: Run all test queries with no feedback\n",
    "    2. Generate synthetic feedback based on reference answers (if provided)\n",
    "    3. Second round: Run the same queries with feedback-enhanced retrieval\n",
    "    4. Compare results between rounds to quantify feedback impact\n",
    "    \n",
    "    Args:\n",
    "        pdf_path (str): Path to the PDF document used as the knowledge base\n",
    "        test_queries (List[str]): List of test queries to evaluate system performance\n",
    "        reference_answers (List[str], optional): Reference/gold standard answers for evaluation\n",
    "                                                and synthetic feedback generation\n",
    "        \n",
    "    Returns:\n",
    "        Dict: Evaluation results containing:\n",
    "            - round1_results: Results without feedback\n",
    "            - round2_results: Results with feedback\n",
    "            - comparison: Quantitative comparison metrics between rounds\n",
    "    \"\"\"\n",
    "    print(\"=== Evaluating Feedback Loop Impact ===\")\n",
    "    \n",
    "    # Create a temporary feedback file for this evaluation session only\n",
    "    temp_feedback_file = \"temp_evaluation_feedback.json\"\n",
    "    \n",
    "    # Initialize feedback collection (empty at the start)\n",
    "    feedback_data = []\n",
    "    \n",
    "    # ----------------------- FIRST EVALUATION ROUND -----------------------\n",
    "    # Run all queries without any feedback influence to establish baseline performance\n",
    "    print(\"\\n=== ROUND 1: NO FEEDBACK ===\")\n",
    "    round1_results = []\n",
    "    \n",
    "    for i, query in enumerate(test_queries):\n",
    "        print(f\"\\nQuery {i+1}: {query}\")\n",
    "        \n",
    "        # Process document to create initial vector store\n",
    "        chunks, vector_store = process_document(pdf_path)\n",
    "        \n",
    "        # Execute RAG without feedback influence (empty feedback list)\n",
    "        result = rag_with_feedback_loop(query, vector_store, [])\n",
    "        round1_results.append(result)\n",
    "        \n",
    "        # Generate synthetic feedback if reference answers are available\n",
    "        # This simulates user feedback for training the system\n",
    "        if reference_answers and i < len(reference_answers):\n",
    "            # Calculate synthetic feedback scores based on similarity to reference answer\n",
    "            similarity_to_ref = calculate_similarity(result[\"response\"], reference_answers[i])\n",
    "            # Convert similarity (0-1) to rating scale (1-5)\n",
    "            relevance = max(1, min(5, int(similarity_to_ref * 5)))\n",
    "            quality = max(1, min(5, int(similarity_to_ref * 5)))\n",
    "            \n",
    "            # Create structured feedback entry\n",
    "            feedback = get_user_feedback(\n",
    "                query=query,\n",
    "                response=result[\"response\"],\n",
    "                relevance=relevance,\n",
    "                quality=quality,\n",
    "                comments=f\"Synthetic feedback based on reference similarity: {similarity_to_ref:.2f}\"\n",
    "            )\n",
    "            \n",
    "            # Add to in-memory collection and persist to temporary file\n",
    "            feedback_data.append(feedback)\n",
    "            store_feedback(feedback, temp_feedback_file)\n",
    "    \n",
    "    # ----------------------- SECOND EVALUATION ROUND -----------------------\n",
    "    # Run the same queries with feedback incorporation to measure improvement\n",
    "    print(\"\\n=== ROUND 2: WITH FEEDBACK ===\")\n",
    "    round2_results = []\n",
    "    \n",
    "    # Process document and enhance with feedback-derived content\n",
    "    chunks, vector_store = process_document(pdf_path)\n",
    "    vector_store = fine_tune_index(vector_store, chunks, feedback_data)\n",
    "    \n",
    "    for i, query in enumerate(test_queries):\n",
    "        print(f\"\\nQuery {i+1}: {query}\")\n",
    "        \n",
    "        # Execute RAG with feedback influence\n",
    "        result = rag_with_feedback_loop(query, vector_store, feedback_data)\n",
    "        round2_results.append(result)\n",
    "    \n",
    "    # ----------------------- RESULTS ANALYSIS -----------------------\n",
    "    # Compare performance metrics between the two rounds\n",
    "    comparison = compare_results(test_queries, round1_results, round2_results, reference_answers)\n",
    "    \n",
    "    # Clean up temporary evaluation artifacts\n",
    "    if os.path.exists(temp_feedback_file):\n",
    "        os.remove(temp_feedback_file)\n",
    "    \n",
    "    return {\n",
    "        \"round1_results\": round1_results,\n",
    "        \"round2_results\": round2_results,\n",
    "        \"comparison\": comparison\n",
    "    }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Helper Functions for Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "def calculate_similarity(text1, text2):\n",
    "    \"\"\"\n",
    "    Calculate semantic similarity between two texts using embeddings.\n",
    "    \n",
    "    Args:\n",
    "        text1 (str): First text\n",
    "        text2 (str): Second text\n",
    "        \n",
    "    Returns:\n",
    "        float: Similarity score between 0 and 1\n",
    "    \"\"\"\n",
    "    # Generate embeddings for both texts\n",
    "    embedding1 = create_embeddings(text1)\n",
    "    embedding2 = create_embeddings(text2)\n",
    "    \n",
    "    # Convert embeddings to numpy arrays\n",
    "    vec1 = np.array(embedding1)\n",
    "    vec2 = np.array(embedding2)\n",
    "    \n",
    "    # Calculate cosine similarity between the two vectors\n",
    "    similarity = np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))\n",
    "    \n",
    "    return similarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "def compare_results(queries, round1_results, round2_results, reference_answers=None):\n",
    "    \"\"\"\n",
    "    Compare results from two rounds of RAG.\n",
    "    \n",
    "    Args:\n",
    "        queries (List[str]): Test queries\n",
    "        round1_results (List[Dict]): Results from round 1\n",
    "        round2_results (List[Dict]): Results from round 2\n",
    "        reference_answers (List[str], optional): Reference answers\n",
    "        \n",
    "    Returns:\n",
    "        str: Comparison analysis\n",
    "    \"\"\"\n",
    "    print(\"\\n=== COMPARING RESULTS ===\")\n",
    "    \n",
    "    # System prompt to guide the AI's evaluation behavior\n",
    "    system_prompt = \"\"\"You are an expert evaluator of RAG systems. Compare responses from two versions:\n",
    "        1. Standard RAG: No feedback used\n",
    "        2. Feedback-enhanced RAG: Uses a feedback loop to improve retrieval\n",
    "\n",
    "        Analyze which version provides better responses in terms of:\n",
    "        - Relevance to the query\n",
    "        - Accuracy of information\n",
    "        - Completeness\n",
    "        - Clarity and conciseness\n",
    "    \"\"\"\n",
    "\n",
    "    comparisons = []\n",
    "    \n",
    "    # Iterate over each query and its corresponding results from both rounds\n",
    "    for i, (query, r1, r2) in enumerate(zip(queries, round1_results, round2_results)):\n",
    "        # Create a prompt for comparing the responses\n",
    "        comparison_prompt = f\"\"\"\n",
    "        Query: {query}\n",
    "\n",
    "        Standard RAG Response:\n",
    "        {r1[\"response\"]}\n",
    "\n",
    "        Feedback-enhanced RAG Response:\n",
    "        {r2[\"response\"]}\n",
    "        \"\"\"\n",
    "\n",
    "        # Include reference answer if available\n",
    "        if reference_answers and i < len(reference_answers):\n",
    "            comparison_prompt += f\"\"\"\n",
    "            Reference Answer:\n",
    "            {reference_answers[i]}\n",
    "            \"\"\"\n",
    "\n",
    "        comparison_prompt += \"\"\"\n",
    "        Compare these responses and explain which one is better and why.\n",
    "        Focus specifically on how the feedback loop has (or hasn't) improved the response quality.\n",
    "        \"\"\"\n",
    "\n",
    "        # Call the OpenAI API to generate a comparison analysis\n",
    "        response = client.chat.completions.create(\n",
    "            model=\"meta-llama/Llama-3.2-3B-Instruct\",\n",
    "            messages=[\n",
    "                {\"role\": \"system\", \"content\": system_prompt},\n",
    "                {\"role\": \"user\", \"content\": comparison_prompt}\n",
    "            ],\n",
    "            temperature=0\n",
    "        )\n",
    "        \n",
    "        # Append the comparison analysis to the results\n",
    "        comparisons.append({\n",
    "            \"query\": query,\n",
    "            \"analysis\": response.choices[0].message.content\n",
    "        })\n",
    "        \n",
    "        # Print a snippet of the analysis for each query\n",
    "        print(f\"\\nQuery {i+1}: {query}\")\n",
    "        print(f\"Analysis: {response.choices[0].message.content[:200]}...\")\n",
    "    \n",
    "    return comparisons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation of the feedback loop (Custom Validation Queries)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=== Evaluating Feedback Loop Impact ===\n",
      "\n",
      "=== ROUND 1: NO FEEDBACK ===\n",
      "\n",
      "Query 1: What is a neural network and how does it function?\n",
      "Extracting text from PDF...\n",
      "Chunking text...\n",
      "Created 42 text chunks\n",
      "Creating embeddings for chunks...\n",
      "Added 42 chunks to the vector store\n",
      "\n",
      "=== Processing query with feedback-enhanced RAG ===\n",
      "Query: What is a neural network and how does it function?\n",
      "Generating response...\n",
      "\n",
      "=== Response ===\n",
      "Based on the provided context, a neural network is a type of deep neural network that is particularly effective for processing data. The context does not provide a detailed explanation of how a neural network functions, but it does mention that neural networks are inspired by the structure and function of the human brain.\n",
      "\n",
      "However, it can be inferred that a neural network is a complex system that uses multiple layers to analyze data. The context mentions that deep learning is a subfield of machine learning that uses artificial neural networks with multiple layers (deep neural networks) to analyze data.\n",
      "\n",
      "In the context of the provided text, neural networks are used in various applications such as image recognition, natural language processing, and speech recognition. They are also used in tasks like object detection, facial recognition, and medical image analysis.\n",
      "\n",
      "While the context does not provide a comprehensive explanation of how a neural network functions, it can be inferred that neural networks are designed to learn patterns and structures in the data through a process of trial and error, similar to reinforcement learning. However, the exact mechanism of how neural networks function is not explicitly stated in the provided context.\n",
      "\n",
      "=== ROUND 2: WITH FEEDBACK ===\n",
      "Extracting text from PDF...\n",
      "Chunking text...\n",
      "Created 42 text chunks\n",
      "Creating embeddings for chunks...\n",
      "Added 42 chunks to the vector store\n",
      "Fine-tuning index with high-quality feedback...\n",
      "Added enhanced content from feedback: What is a neural network and how does it function?...\n",
      "Fine-tuned index now has 43 items (original: 42)\n",
      "\n",
      "Query 1: What is a neural network and how does it function?\n",
      "\n",
      "=== Processing query with feedback-enhanced RAG ===\n",
      "Query: What is a neural network and how does it function?\n",
      "Adjusting relevance scores based on feedback history...\n",
      "  Document 1: Adjusted score from 0.8386 to 1.0902 based on 1 feedback(s)\n",
      "  Document 4: Adjusted score from 0.6162 to 0.8010 based on 1 feedback(s)\n",
      "  Document 5: Adjusted score from 0.6023 to 0.7830 based on 1 feedback(s)\n",
      "Generating response...\n",
      "\n",
      "=== Response ===\n",
      "Based on the provided context, a neural network is a complex system that uses multiple layers to analyze data. It is inspired by the structure and function of the human brain and is particularly effective for processing data. Neural networks are used in various applications such as image recognition, natural language processing, and speech recognition.\n",
      "\n",
      "The context does not provide a detailed explanation of how a neural network functions, but it can be inferred that neural networks are designed to learn patterns and structures in the data through a process of trial and error. This process is similar to reinforcement learning, where the neural network receives feedback in the form of rewards or penalties, allowing it to adjust its parameters and improve its performance over time.\n",
      "\n",
      "Neural networks are composed of multiple layers, including convolutional layers, recurrent layers, and others. These layers work together to analyze the input data and make predictions or take actions. The exact mechanism of how neural networks function is not explicitly stated in the provided context, but it is clear that they are a powerful tool for analyzing and processing complex data.\n",
      "\n",
      "In the context of the provided text, neural networks are used in various applications such as:\n",
      "\n",
      "* Image recognition\n",
      "* Natural language processing\n",
      "* Speech recognition\n",
      "* Object detection\n",
      "* Facial recognition\n",
      "* Medical image analysis\n",
      "\n",
      "Overall, neural networks are a complex and powerful tool for analyzing and processing data, and their applications continue to expand across various industries and domains.\n",
      "\n",
      "=== COMPARING RESULTS ===\n",
      "\n",
      "Query 1: What is a neural network and how does it function?\n",
      "Analysis: Comparing the two responses, the feedback-enhanced RAG response is significantly better than the standard RAG response. Here's a breakdown of the improvements:\n",
      "\n",
      "1. **Relevance to the query**: Both res...\n"
     ]
    }
   ],
   "source": [
    "# AI Document Path\n",
    "pdf_path = \"data/AI_Information.pdf\"\n",
    "\n",
    "# Define test queries\n",
    "test_queries = [\n",
    "    \"What is a neural network and how does it function?\",\n",
    "\n",
    "    #################################################################################\n",
    "    ### Commented out queries to reduce the number of queries for testing purposes ###\n",
    "    \n",
    "    # \"Describe the process and applications of reinforcement learning.\",\n",
    "    # \"What are the main applications of natural language processing in today's technology?\",\n",
    "    # \"Explain the impact of overfitting in machine learning models and how it can be mitigated.\"\n",
    "]\n",
    "\n",
    "# Define reference answers for evaluation\n",
    "reference_answers = [\n",
    "    \"A neural network is a series of algorithms that attempt to recognize underlying relationships in a set of data through a process that mimics the way the human brain operates. It consists of layers of nodes, with each node representing a neuron. Neural networks function by adjusting the weights of connections between nodes based on the error of the output compared to the expected result.\",\n",
    "\n",
    "    ############################################################################################\n",
    "    #### Commented out reference answers to reduce the number of queries for testing purposes ###\n",
    "\n",
    "#     \"Reinforcement learning is a type of machine learning where an agent learns to make decisions by performing actions in an environment to maximize cumulative reward. It involves exploration, exploitation, and learning from the consequences of actions. Applications include robotics, game playing, and autonomous vehicles.\",\n",
    "#     \"The main applications of natural language processing in today's technology include machine translation, sentiment analysis, chatbots, information retrieval, text summarization, and speech recognition. NLP enables machines to understand and generate human language, facilitating human-computer interaction.\",\n",
    "#     \"Overfitting in machine learning models occurs when a model learns the training data too well, capturing noise and outliers. This results in poor generalization to new data, as the model performs well on training data but poorly on unseen data. Mitigation techniques include cross-validation, regularization, pruning, and using more training data.\"\n",
    "]\n",
    "\n",
    "# Run the evaluation\n",
    "evaluation_results = evaluate_feedback_loop(\n",
    "    pdf_path=pdf_path,\n",
    "    test_queries=test_queries,\n",
    "    reference_answers=reference_answers\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "########################################\n",
    "# # Run a full RAG workflow\n",
    "########################################\n",
    "\n",
    "# # Run an interactive example\n",
    "# print(\"\\n\\n=== INTERACTIVE EXAMPLE ===\")\n",
    "# print(\"Enter your query about AI:\")\n",
    "# user_query = input()\n",
    "\n",
    "# # Load accumulated feedback\n",
    "# all_feedback = load_feedback_data()\n",
    "\n",
    "# # Run full workflow\n",
    "# result = full_rag_workflow(\n",
    "#     pdf_path=pdf_path,\n",
    "#     query=user_query,\n",
    "#     feedback_data=all_feedback,\n",
    "#     fine_tune=True\n",
    "# )\n",
    "\n",
    "########################################\n",
    "# # Run a full RAG workflow\n",
    "########################################"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualizing Feedback Impact"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "=== FEEDBACK IMPACT ANALYSIS ===\n",
      "\n",
      "Query 1: What is a neural network and how does it function?\n",
      "\n",
      "Analysis of feedback impact:\n",
      "Comparing the two responses, the feedback-enhanced RAG response is significantly better than the standard RAG response. Here's a breakdown of the improvements:\n",
      "\n",
      "1. **Relevance to the query**: Both responses address the query, but the feedback-enhanced RAG response provides a more comprehensive and detailed explanation of what a neural network is and how it functions. It also provides more specific examples of applications, which shows a better understanding of the topic.\n",
      "\n",
      "2. **Accuracy of information**: The feedback-enhanced RAG response is more accurate, as it provides a clear and concise explanation of how neural networks function, including the concept of trial and error, reinforcement learning, and the role of feedback. The standard RAG response is less accurate, as it only mentions that neural networks are inspired by the human brain and that they are used in various applications, but does not provide a clear explanation of how they function.\n",
      "\n",
      "3. **Completeness**: The feedback-enhanced RAG response is more complete, as it provides a detailed explanation of the components of a neural network, including convolutional layers, recurrent layers, and others. The standard RAG response only mentions that neural networks are used in various applications, but does not provide a clear explanation of how they function.\n",
      "\n",
      "4. **Clarity and conciseness**: The feedback-enhanced RAG response is more concise and clear, as it provides a clear and concise explanation of how neural networks function and their applications. The standard RAG response is less concise and less clear, as it only mentions that neural networks are inspired by the human brain and that they are used in various applications, but does not provide a clear explanation of how they function.\n",
      "\n",
      "The feedback loop has significantly improved the response quality by:\n",
      "\n",
      "* Providing a clear and concise explanation of how neural networks function\n",
      "* Providing more specific examples of applications\n",
      "* Addressing the limitations of the standard RAG response (e.g., lack of clarity and concision)\n",
      "* Providing a more accurate explanation of the components of a neural network\n",
      "\n",
      "The feedback loop has helped to identify the areas where the standard RAG response was lacking and has provided a more comprehensive and detailed explanation of the topic. This is a clear example of how a feedback loop can improve the quality of a response.\n",
      "\n",
      "--------------------------------------------------\n",
      "\n",
      "\n",
      "Response length comparison (proxy for completeness):\n",
      "Round 1: 1256.0 chars\n"
     ]
    }
   ],
   "source": [
    "# Extract the comparison data which contains the analysis of feedback impact\n",
    "comparisons = evaluation_results['comparison']\n",
    "\n",
    "# Print out the analysis results to visualize feedback impact\n",
    "print(\"\\n=== FEEDBACK IMPACT ANALYSIS ===\\n\")\n",
    "for i, comparison in enumerate(comparisons):\n",
    "    print(f\"Query {i+1}: {comparison['query']}\")\n",
    "    print(f\"\\nAnalysis of feedback impact:\")\n",
    "    print(comparison['analysis'])\n",
    "    print(\"\\n\" + \"-\"*50 + \"\\n\")\n",
    "\n",
    "# Additionally, we can compare some metrics between rounds\n",
    "round_responses = [evaluation_results[f'round{round_num}_results'] for round_num in range(1, len(evaluation_results) - 1)]\n",
    "response_lengths = [[len(r[\"response\"]) for r in round] for round in round_responses]\n",
    "\n",
    "print(\"\\nResponse length comparison (proxy for completeness):\")\n",
    "avg_lengths = [sum(lengths) / len(lengths) for lengths in response_lengths]\n",
    "for round_num, avg_len in enumerate(avg_lengths, start=1):\n",
    "    print(f\"Round {round_num}: {avg_len:.1f} chars\")\n",
    "\n",
    "if len(avg_lengths) > 1:\n",
    "    changes = [(avg_lengths[i] - avg_lengths[i-1]) / avg_lengths[i-1] * 100 for i in range(1, len(avg_lengths))]\n",
    "    for round_num, change in enumerate(changes, start=2):\n",
    "        print(f\"Change from Round {round_num-1} to Round {round_num}: {change:.1f}%\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv-new-specific-rag",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
